Using Hadoop as a Grid Storage Element
نویسنده
چکیده
Hadoop is an open-source data processing framework that includes a scalable, faulttolerant distributed file system, HDFS. Although HDFS was designed to work in conjunction with Hadoop’s job scheduler, we have re-purposed it to serve as a grid storage element by adding GridFTP and SRM servers. We have tested the system thoroughly in order to understand its scalability and fault tolerance. The turn-on of the Large Hadron Collider (LHC) in 2009 poses a significant data management and storage challenge; we have been working to introduce HDFS as a solution for data storage for one LHC experiment, the Compact Muon Solenoid (CMS).
منابع مشابه
Smart Grid Unit Commitment with Considerations for Pumped Storage Units Using Hybrid GA-Heuristic Optimization Algorithm
A host of technologies has been developed to achieve these aims of the smart grid. Some of these technologies include plug-in electric vehicle, demand response program, energy storage system and renewable distributed generation. However, the integration of the smart grid technologies in the power system operation studies such as economic emission unit commitment problem causes two major challen...
متن کاملThe Study of Hadoop-based Architecture for Power Quality Monitoring Cloud Model
The proposition of Power Quality Intelligent Information System provide an effect ive platform for the monitoring, analysis and processing to all kinds of power quality problems in Smart Grid.However,with the acceleration of Smart Grid construction,continue increasing of the amount of power quality monitoring sites and gradually improving of the system,various types of monitoring and calculatin...
متن کاملUsing Hadoop File System and MapReduce in a small/medium Grid site
Data storage and data access represent the key of CPU-intensive and data-intensive high performance Grid computing. Hadoop is an open-source data processing framework that includes fault-tolerant and scalable distributed data processing model and execution environment, named MapReduce, and distributed File System, named Hadoop distributed File System (HDFS). HDFS was deployed and tested within ...
متن کاملInvestigation of Storage Options for Scientific Computing on Grid and Cloud Facilities
In recent years, several new storage technologies, such as Lustre, Hadoop, OrangeFS, and BlueArc, have emerged. While several groups have run benchmarks to characterize them under a variety of configurations, more work is needed to evaluate these technologies for the use cases of scientific computing on Grid clusters and Cloud facilities. This paper discusses our evaluation of the technologies ...
متن کاملPower Management in a Utility Connected Micro-Grid with Multiple Renewable Energy Sources
As an efficient alternative to fossil fuels, renewable energy sources have attained great attention due to their sustainable, cost-effective, and environmentally friendly characteristic. However, as a deficiency, renewable energy sources have low reliability because of their non-deterministic and stochastic generation pattern. The use of hybrid renewable generation systems along with the storag...
متن کامل